COE GSA '10

University of Maryland College of Education Graduate Student Research Conference 2010

Opening Plenary -
"Exploring connectivity, sparking creativity, working through crisis: Students speak out! Featuring Dr. Connie North & students and teachers from Prince George's & Montgomery County Schools."
9am-10:15am

Four inquiry-to-action group members:

• Nellie Jackson, teaches English at St. Mary's Ryken, a private, college preparatory, Catholic high school in Southern Maryland.
• Connie North, teaches diversity and qualitative research methods courses in the Dept. of Curriculum and Instruction at the University of Maryland, College Park.
• Beth Novick, teaches sixth-grade English language arts at Greenbelt Elementary, a Prince Georges County public school located northeast of D.C.
• Candace Slobodnik, is a social studies department chair and teacher at Crossland, a Prince Georges public comprehensive high school located southesast of D.C.

Two of Nellie Jackson's former students at Don Bosco Cristo Rey (Takoma Park, MD) shared their visions for education.

• "I can't tell you what made the difference, but..."

Six International Baccalaureate students (first cohort from Crossland High) shared their visions for education.

• First student: - realistic vision of a public high school in Prince George's County (projected drawing). Described a school that seemed far from his reach - where paper towels were available and soap dispensers aren't missing.
• Second student - want a school where the conditions of our books doesn't concern me; I want a school where students want more than stuff. I want a school where no one looks down on other people.
• Third student - shared vision and a quilt their class has created. Perfect education means a chance for everyone to succeed. Sometimes teachers get a check thinking they made a difference in the lives of kids but they didn't. We created this quilt to demonstrate our ideas but were limited by our resources. Colors represent school and represent what our school does for us.
• Fourth student - read a poem entitled "Our perfect school."
• Fifth student - introduced the next group of students and teacher

Nellie Jackson's students from St. Mary's Ryken school

• First student - explained a diagram he created. It's not that the right information isn't being taught, it's that the right information is being taught wrong. Too many schools across the nation teach to the test, students forget the information the next day, and the school system fails because students can't apply the knowledge to anything important in the world.
• Second student - talked about Nellie introducing them to the poem "Why I won't graduate." After a decade and a half of testing, many people can't re-imagine school without high-stakes test.
• Third and fourth student - four words that describe why the ideal learning environment doesn't exist: "No Child Left Behind." The question really is, "How do we fix this?" From the student's perspective, the answer is simple: smaller classes, more teachers, students separated by learning style and appropriate matched with teacher. "A teacher is a lens between the information and the student." Teacher should have the flexibility to adjust that lens for each student.

Beth Novak's students from Greenbelt Elementary School

• 99 kids shared their vision of the perfect school. Ten students collaborated to create a video. Viewed the video retroactively to determine what their message really was:
  • there are some bad things in schools and we need help changing them
  • kids that are in school should be able to have fun and learn at the same time. they give respect back when they get it.
• Piano music by one of the 6th graders, movie directed by one of the students.
• "some kids don't like to sit in a chair for a long time"
• "laptops - kids want them"
• "we spend more time in class than outside"
• "we need time to clear our minds"
• Try not to have a good time...this is supposed to be educational (Lucy from Peanuts)
• "know about my problems"
• "We have the potential to be president, and yet we don't have a voice of our own."
• "some meats in our school lunches don't meet fast-food standards."
• "We must stay active so our brains must be fully active and ready to learn."
• "everyone has their own personal curriculum, with stuff we like and are interested in."

Paper Presentation Session
"Promises and Perils of Educator Incentives: A formative Evaluation of the Teacher Incentive Fund (TIF) Program in the Prince Georges County (MD) Public School"
10:15-11:45am

Presenters: Amy Dougherty, Laura Hyde, Cara Jackson
Discussants: Dr. Betty Malen

Educational Incentives

• Part of a broad set of human capital reforms
  • Theory: improving the ability and/or the willingness of educators to provide effective instructional opportunities will result in improved student outcomes
• Long history in education
• Incentives can be used to affect;
  • supply and distribution of educators
  • performance (effort and PD)
• New initiatives fueled by federal TIF
  • differentiated levels of compensation for administrators and instructional staff in high need schools
• Like terms - pay for performance, performance pay

This Study

• Product of partnership between PGCPS and the Dept. of Ed Policy at UMD
• Formative evaluation of the first year of implementation of FIRST
  • theoretical and empirical literature
  • field research
• Our research demonstrates that even when an educator incentive program reflects "lessons learned" from the literature, challenges persist
• Reconstructed the planning year
• Field research in schools

Review of the Literature in a nutshell

• Descriptive literature - variability in terms of:
  • goals,
  • eligibility criteria
  • nature of award
  • measures used to determine the awards
• impact literature -- inconsistent, incomplete, and plagued by methodological concerns
• design and implementation literature - four challenges:
  • stakeholder support
  • measurement
  • capacity
  • alignment

Review of the literature: the challenges

• Stakeholder support - support for incentive programs influence by:
  • opportunity to participate in design process (Heneman, 1998)
  • size, salience, and attainability (Hatry et al, 1994)
  • perceived transparency/fairness of the program (Odden et al., 2001)
• Measurement - educator incentives require accurate, dependable, and credible measures of performance
  • Key Issues -
    • objectivity
    • alignment between measures and goals
    • transparency
    • accuracy
    • individual vs. collective contributions
• Capacity - issues of district capacity include:
  • sustaining funds for the initiative
  • acquiring and building adequate infrastructure for data systems and data dissemination
  • ensuring adequate program administration and oversight
• issues of school capacity include:
  • site administrators' capacity to evaluate teachers and serve as instructional leaders
  • teachers' capacity to improve their teaching practice and realize the goals of the program
• Alignment - compensation systems should be aligned with:
  • overarching goals,
  • human resource strategies,
  • educator evaluation systems,
  • features of the work,
  • school improvement approaches.

PG County Study Contextual Information

• PGCPS is a large diverse school district with a history of chronically under-performing schools
• PGCPS faces significant staffing challenges
• TIF provided district leaders with the opportunity to:
  • attract and retain quality teachers and administrators
  • improve educator performance
  • support a new teacher evaluation system
  • enhance the image of the district
• FIRST
  • focused on 42 high=needs schools in the grant proposal
  • PGCPS built in planning year and phased-in implementation
    • 12 schools in pilot
    • gradual roll-out to other schools over the next three years
  • Eligible personned included
    • teachers of record (though not all eligible teachers are eligible for each component of the program) and
    • site administrators (though not sp. ed. program directors)
• Award Components
  • teachers eligible - $10k,
  • assistant principle - $11,5 k
  • principal - $12k

Study Methods and Data Sources

• Study recaptured the planning year and spanned the pilot year
• Field research included data collection at the district level and case studies of three schools
• Data sources included
  • documents
  • observations
  • interviews
• Data analysis was an ongoing, progressive process
• Process included multiple checks for bias and error

Findings (organized by the four challenges in the lit review)

• PGCPS used multiple strategies to secure stakeholder support
  • a promising program design with voluntary participation
  • multiple avenues for awards to attract and retain educators
  • collaborative structures and reasonable time frames for planning nad implementation
  • persuasive appeals to potential participants
  • multiple opportunities to learn about the program
  • ongoing program adjustments to deflect criticism
• PGCPS encountered persistent and emergent challenges in retaining stakeholder support
  • concerns about ineligibility of some personnel
  • vulnerable district-union partnerships threatened by an economic crisis
  • teachers and administators mixed responses to the initiative
  • confounding trade-offs in implementation that strained capacity, highlighted measurement issues, and precipitated new problems
• PGCPS addressed measurement issues in recommended ways
  • multiple measures to evaluate teacher and administrator performance
  • broadly appealing standards-based teacher evaluation system
  • individual and collective measures of teaching contributions to student performance
  • mandatory training to explain measures
• PGCPS encountered persistent and emergent challenges with measurement
  • limited understanding/acceptance of VAM and growth-over-time-model
  • multiple missteps in implementing FFT-based evaluation system
  • reduction in requirements and pro-rated awards for incomplete administrator portfolios
• PGCPS addressed district capactiy
  • secured the TIF grant
  • established FIRST office
  • hired external consultants
• PGCPS addressed site capacity
  • introduced the FFT-based evaluation protocol
  • provided professional development
  • included leadership projects
• PGCPS encountered persistent and emergent challenges with capacity at the district and site level
  • a heavy reliance on external consultants
  • inadequate data systems
  • frequent turnover of personnel
  • lack of integration of the FIRST office into organization
  • insufficient time and training to implement the FFT-based evaluation system
  • limited attention to how FIRST fits with other initiatives
• PGCPS addressed alignment with
  • broad district goals
  • organizational structures
  • plans to implement FFT-based evaluation system
• PGCPS encountered persistent and emergent challenges with alignment
  • little to no impact on schools' focus on student performance
  • negligible impact on human capital
  • little connection with other parts of organization and largely unsupervised
  • uneven understanding about how FIRST fits with other reform initiatives
  • lack of alignment between program eligibility and site perceptions of who contributes to student performance

Conclusions and Implications:

• Case extends extant literature by
  • directing attention to full scope of challenges
  • illustrating interrelated nature of challenges and complications that result from addressing any one challenge
• Case holds implications for policy by
  • illustrating the necessity for ongoing opportunities for evaluation and adjustment to address ongoing and emergent challenges
  • highlighting the full range of requisite fiscal, organizational, and human resources at the district and site level
  • demonstrating the importance of addressing site educators' conceptions of fair practice
• Longitudinal and comparative case studies of educator incentives to expand and test our understanding of multiple, inter-related challenges
• Longitudinal and comparative studies of program's impact on educator recruitment, retention and distribution, and educator and student performance.

questions
- teachers in more than one building
- teachers ideas about parallels with nclb
- emergent challenges - flexibility of grant and county resources
- impact of the size of district

discussion
- danielson framework applied to teacher incentive programs

oh!
- importance of documenting complexity of events, even though it may not be a unique finding

Paper presentations
3:00-4:30pm